The present subject matter relates generally to synthetic apertures for visible imaging. More specifically, the present invention relates to synthetic apertures for visible imaging as a promising approach to achieve sub-diffraction resolution in long distance imaging.
Imaging objects from large standoff distances is a requirement in many computer vision and imaging applications such as surveillance and remote sensing. In these scenarios, the imaging device is sufficiently far away from the object that imaging resolution is fundamentally limited not by magnification, but rather by the diffraction of light at the limiting aperture of the imaging system: using a lens with a larger aperture will lead to increased spatial resolution. Physically increasing the aperture of the lens, by building a larger lens, results in expensive, heavy, and bulky optics and mechanics. A number of techniques have been proposed to improve spatial resolution for various imaging systems, including refractive telescopes (1-6), holography (7-11) and incoherent super-resolution (12-16).
The resolution of an imaging system is proportional to both the lens aperture size and the wavelength of the electromagnetic spectrum used. In long wavelength regimes (such as radar), the direct coupling between image resolution and aperture size can be mitigated using synthetic aperture radar (SAR) techniques. SAR improves radio imaging resolution by capturing multiple measurements of a static object using a mobile recording platform, such as an airplane or satellite as shown in the diagram of FIG. 1A. For SAR, the resolution is determined by the synthetic aperture size, which can be many orders of magnitude larger than the physical aperture size. Stitching together multiple radar returns is possible because the full complex field (amplitude and phase) is directly measured by the antenna with picosecond timing resolution.
As noted above, stitching together multiple radar returns from a SAR technique is possible because the amplitude and phase is measured with picosecond timing resolution. To make a comparable measurement using visible light, a detector would have to continuously record information with a time resolution greater than one femtosecond, a requirement well beyond the capabilities of modern devices. As such, current camera sensors record only the intensity of the incoming optical field and all phase information is lost.
Fourier ptychography (FP) has emerged as a powerful tool to improve spatial resolution in microscopy. In FP, multiple low-resolution images under different illumination angles are captured and stitched together (17-27). Redundancy between measurements permits computational recovery of the missing phase information (18,25, 28-31). Fourier ptychography creates a synthetic aperture by sampling a diverse set of regions in Fourier space. Unlike holography, FP does not require the use of a reference beam to encode phase information. The phase of the complex field is recovered computationally in post-processing. FP has found much of its success in microscopy. Early efforts by Kirkland et al. (59, 60) demonstrated that multiple images recorded with different incident beam tilts could be used to effectively double image resolution. Zheng et al. (17) provided a complete framework for FP microscopy and demonstrated wide-field, high-resolution imaging. Subsequent research has improved the quality of FP reconstructions by characterizing the pupil function (18), digitally removing optical aberrations (19), and refocusing the recovered image postcapture (20). FP microscopy (where the illumination direction is varied) inherently assumes that the sample may be modeled as a thin object. Extensions for thick biological samples (21-23) have been proposed at the expense of increased computational complexity.
At the heart of FP is the requirement to recover the phase of the light field at the aperture plane of the lens, which subsequently provides knowledge of the field at the object plane. Phase retrieval is also an important step in standard and many of the techniques used in FP are borrowed from these earlier efforts.
In general, closed form solutions for recovering phase information require prohibitively large datasets to be practical (61-63). Iterative solutions are thus preferred for ptychographic reconstruction. Many FP reconstruction algorithms are based on the iterative update schemes first proposed by Gerchberg and Saxton (28) and Fienup (29). Maiden and Rodenburg (30) introduced the ePIE technique to jointly estimate the field at the detector and the probe used for illumination. Ou et al. (18) adapted ePIE for use in FP whereby the pupil function is jointly estimated with the field at the aperture plane. Experimental robustness of various phase retrieval algorithms were characterized by Yeh et al. (31) who conclude that minimizing the error in amplitude and using second-order gradient descent methods provide the best results. The phase retrieval algorithm used by Tian et al. (25), which incorporates the pupil update step of (18) and uses the second-order Newton's method as the numerical solver, serves as the base of our proposed algorithm. Although the objective function of the reconstruction framework in (25) minimizes intensities and not amplitudes, our experiments have resulted in good reconstruction quality.
Adapting the technique to long-distance imaging requires two important modifications of previous FP microscopy implementations. First, the separation distance between object and camera increases by orders of magnitude. Second, a reflection imaging geometry must be used so that illumination source and camera are placed on the same side of the object. Dong et al. (20) and Holloway et al. (32) succeeded in the first task, scaling up the working distance to 0.7 and 1.5 meters, respectively. Reflective FP microscopy setups have been proposed to fulfill the second task (33-35). However, these systems either require small working distances (34, 35), or exhibit limited reconstruction performance (33).
In FIGS. 1B and 1C, a comparison with existing FP implementations is shown. Previous works have relied on smooth objects and are loosely represented by the transmissive dataset adapted from (32) shown in FIG. 1B. An example dataset of a diffuse object collected in a reflection mode geometry is shown in FIG. 1C. The immediate difference between the two datasets is the random phase associated with diffuse objects effectively spreads out information across the entire Fourier domain. The difference in Fourier patterns is evident in the captured images taken from the same locations in both modalities. As a consequence of the random phase, the spatial information is obfuscated by the resultant speckle.
Tippie et al. (11) proposes a synthetic aperture holographic setup in which the authors experimentally demonstrated synthetic aperture off-axis holographic capture of diffuse objects at a large stand-off distance. Our approach can be interpreted as a reference-free extension of synthetic aperture holography in which computational reconstruction algorithms are used in place of interferometric capture, resulting in more stable implementations and widening the variety of application scenarios that could benefit from the approach. Beck et al. (36) proposes an optical synthetic aperture approach that extends SAR techniques into optical wavelengths in the near-infrared regime of the electromagnetic spectrum. To record phase measurements, the object is raster scanned by moving an aperture. The return signal is then demodulated using a reference signal to reduce the frequency to approximately 100 kHz, which can be sampled with commercially available ADCs.